<html>
<head><meta charset="utf-8"><title>Performance regressions in 1.48.0 · general · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/index.html">general</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html">Performance regressions in 1.48.0</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="217397065"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217397065" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> marmeladema <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217397065">(Nov 20 2020 at 12:57)</a>:</h4>
<p>Hello!<br>
Has anyone else noticed performance regressions with last stable version? My macro benchmarks shows up to 13% regressions. I will _try_ to isolate the issue.<br>
But I mostly wondered if anyone else had notice a similar thing?</p>



<a name="217399125"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217399125" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Philipp Korber <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217399125">(Nov 20 2020 at 13:17)</a>:</h4>
<p>Compile or runtime performance?</p>



<a name="217402819"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217402819" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> marmeladema <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217402819">(Nov 20 2020 at 13:51)</a>:</h4>
<p>Runtime performance of a rust project built by rustc</p>



<a name="217405154"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217405154" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217405154">(Nov 20 2020 at 14:10)</a>:</h4>
<p>if it's runtime performance, I am confused by the mention of macros</p>



<a name="217406211"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217406211" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> marmeladema <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217406211">(Nov 20 2020 at 14:19)</a>:</h4>
<p>Sorry for the confusion. I meant macro benchmark as opposed to micro benchmark</p>



<a name="217406319"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217406319" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> marmeladema <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217406319">(Nov 20 2020 at 14:20)</a>:</h4>
<p>Now that I reread it with your perspective, I can see how confusing that can be^^</p>



<a name="217406605"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217406605" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217406605">(Nov 20 2020 at 14:23)</a>:</h4>
<p>Isolating would be great.</p>



<a name="217421017"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217421017" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> marmeladema <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217421017">(Nov 20 2020 at 16:08)</a>:</h4>
<p>Is there a bisecting script/tool that I could re-use to build my project with all nightly versions between 1.47 to 1.48?</p>



<a name="217421068"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217421068" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> lqd <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217421068">(Nov 20 2020 at 16:09)</a>:</h4>
<p>there is, <a href="https://github.com/rust-lang/cargo-bisect-rustc/">https://github.com/rust-lang/cargo-bisect-rustc/</a></p>



<a name="217421107"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217421107" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> marmeladema <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217421107">(Nov 20 2020 at 16:09)</a>:</h4>
<p>ah! nice!</p>



<a name="217422495"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217422495" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> marmeladema <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217422495">(Nov 20 2020 at 16:17)</a>:</h4>
<p>Thank you very much! That looks really helpful. I will first try to reproduce the slowdown locally with a script</p>



<a name="217436736"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217436736" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> marmeladema <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217436736">(Nov 20 2020 at 17:58)</a>:</h4>
<p>Ok, I can reproduce locally :)</p>



<a name="217439558"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217439558" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> marmeladema <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217439558">(Nov 20 2020 at 18:20)</a>:</h4>
<p>So I have written my <a href="http://bisect.sh">bisect.sh</a> script and tried to use <code>cargo-bisect-rustc</code> with the following commits:</p>
<ul>
<li>start = 1.47.0 = <a href="https://github.com/rust-lang/rust/commit/18bf6b4f01a6feaf7259ba7cdae58031af1b7b39">18bf6b4f01a6feaf7259ba7cdae58031af1b7b39</a></li>
<li>end = 1.48.0 = <a href="https://github.com/rust-lang/rust/commit/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4">7eac88abb2e57e752f3302f02be5f3ce3d7adfb4</a><br>
and here is what I got:</li>
</ul>
<div class="codehilite"><pre><span></span><code>$ cargo-bisect-rustc --by-commit --start=18bf6b4f01a6feaf7259ba7cdae58031af1b7b39 --end=7eac88abb2e57e752f3302f02be5f3ce3d7adfb4 --script=./bisect.sh
bisecting ci builds
starting at 18bf6b4f01a6feaf7259ba7cdae58031af1b7b39, ending at 7eac88abb2e57e752f3302f02be5f3ce3d7adfb4
cloning rust repository
fetching (via local git) commits from 18bf6b4f01a6feaf7259ba7cdae58031af1b7b39 to 7eac88abb2e57e752f3302f02be5f3ce3d7adfb4
opening existing repository at &quot;rust.git&quot;
refreshing repository
looking up first commit
looking up second commit
checking that commits are by bors and thus have ci artifacts...
finding bors merge commits
ERROR: failed during attempt to create/access local git repository: Expected author Brian Anderson to be bors for beb9a0dfc52ebda4f8db4e5d439e08e4f3a43a39
</code></pre></div>



<a name="217439889"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217439889" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> marmeladema <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217439889">(Nov 20 2020 at 18:23)</a>:</h4>
<p>Those commits are matching with the tags on github: <a href="https://github.com/rust-lang/rust/tags">https://github.com/rust-lang/rust/tags</a></p>



<a name="217441026"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217441026" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217441026">(Nov 20 2020 at 18:33)</a>:</h4>
<p>I would not try to pass explicit start/end, just bisect normally.</p>



<a name="217441149"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217441149" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> marmeladema <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217441149">(Nov 20 2020 at 18:34)</a>:</h4>
<p>I have never used that tool, so I don't really know what is the "normal" way to use it. I just followed to guide provided on the repo: <a href="https://github.com/rust-lang/cargo-bisect-rustc/blob/master/TUTORIAL.md">https://github.com/rust-lang/cargo-bisect-rustc/blob/master/TUTORIAL.md</a></p>



<a name="217441267"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217441267" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> lqd <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217441267">(Nov 20 2020 at 18:35)</a>:</h4>
<p>yeah try with the start and end dates like shown there</p>



<a name="217441672"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217441672" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> marmeladema <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217441672">(Nov 20 2020 at 18:39)</a>:</h4>
<p>Ok, used <code>cargo bisect-rustc --script=./bisect.sh --end=2020-11-16</code></p>



<a name="217441735"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217441735" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> marmeladema <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217441735">(Nov 20 2020 at 18:39)</a>:</h4>
<p>Since rustup says <code>rustc 1.48.0 (7eac88abb 2020-11-16)</code></p>



<a name="217442393"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217442393" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217442393">(Nov 20 2020 at 18:45)</a>:</h4>
<p>I mean you don't need to pass explicit start/end, it should be fine without that</p>



<a name="217442544"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217442544" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> marmeladema <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217442544">(Nov 20 2020 at 18:46)</a>:</h4>
<p>Ok, well it has started with --end=2020-11-16 so i'll keep it running like that for now</p>



<a name="217442578"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217442578" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> marmeladema <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217442578">(Nov 20 2020 at 18:46)</a>:</h4>
<p>each build/test take 5/6min so I think it will run for a while</p>



<a name="217442579"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217442579" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> lqd <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217442579">(Nov 20 2020 at 18:46)</a>:</h4>
<p>as long it's not using commits it's not going to change much</p>



<a name="217442695"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217442695" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> marmeladema <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217442695">(Nov 20 2020 at 18:48)</a>:</h4>
<p>Just for my knowledge though, when it says: "RESULT: nightly-2020-11-16, ===&gt; Yes" that means the bug was reproduced by that version?<br>
i.e: that the <a href="http://bisect.sh">bisect.sh</a> script returned != 0 ?</p>



<a name="217442885"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217442885" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> lqd <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217442885">(Nov 20 2020 at 18:49)</a>:</h4>
<p>IIRC yes</p>



<a name="217443029"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217443029" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> marmeladema <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217443029">(Nov 20 2020 at 18:50)</a>:</h4>
<p>Ok thanks :)</p>



<a name="217457639"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217457639" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> marmeladema <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217457639">(Nov 20 2020 at 20:56)</a>:</h4>
<p>Bisection has ended and points to <a href="https://github.com/rust-lang/rust/commit/0d0f6b113047b2cf9afbde990cee30fd5b866469">https://github.com/rust-lang/rust/commit/0d0f6b113047b2cf9afbde990cee30fd5b866469</a></p>



<a name="217457682"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217457682" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> marmeladema <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217457682">(Nov 20 2020 at 20:57)</a>:</h4>
<p>The code that regressed is unfortunately not open source so I am not sure if I will be able to provide a standalone and minimized example to reproduce it</p>



<a name="217457767"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217457767" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> marmeladema <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217457767">(Nov 20 2020 at 20:58)</a>:</h4>
<p>What would be the way forward?</p>



<a name="217459423"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217459423" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Dirkjan Ochtman <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217459423">(Nov 20 2020 at 21:12)</a>:</h4>
<p>Without some way of reproducing the slowdown, getting it fixed seems unlikely</p>



<a name="217459666"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217459666" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> marmeladema <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217459666">(Nov 20 2020 at 21:14)</a>:</h4>
<p>I will do my best, but that won't be easy :/</p>



<a name="217460278"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217460278" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jake Goulding <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217460278">(Nov 20 2020 at 21:20)</a>:</h4>
<p><span class="user-mention" data-user-id="281572">@marmeladema</span> perhaps <a href="https://github.com/shepmaster/rust-mre">https://github.com/shepmaster/rust-mre</a> can help.</p>



<a name="217460297"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217460297" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jake Goulding <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217460297">(Nov 20 2020 at 21:20)</a>:</h4>
<p>There are also automated reduction tools.</p>



<a name="217460642"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217460642" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> marmeladema <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217460642">(Nov 20 2020 at 21:23)</a>:</h4>
<p>Great! Thank you! I will try that in the next couple of days (or next week...)<br>
In the mean time, I have created <a href="https://github.com/rust-lang/rust/issues/79246">https://github.com/rust-lang/rust/issues/79246</a></p>



<a name="217461085"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217461085" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> marmeladema <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217461085">(Nov 20 2020 at 21:27)</a>:</h4>
<p>To be honest, when you look at the numbers in the PR, it's already suspicious, on average -0.7% and +0.3% does not seem like a real improvement.</p>



<a name="217464702"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217464702" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Thom Chiovoloni <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217464702">(Nov 20 2020 at 22:04)</a>:</h4>
<p>Avoiding the system allocator can be a huge win on some platforms. (On macos in particular, due to <code>free</code> being required to check with every registered malloc_zone_t).</p>
<p>The rustc-bench stuff will be on linux when using jemalloc, which won't have those problems and have very cheap allocs.</p>



<a name="217497453"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217497453" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> The 8472 <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217497453">(Nov 21 2020 at 11:18)</a>:</h4>
<p>Not sure which perf results you're looking at, there are much larger wins than -0.7%. The last measurements for that PR can be found here: <a href="https://perf.rust-lang.org/compare.html?start=62dad457bc73804891c6ac9a31f90de19cbb59a3&amp;end=0d0f6b113047b2cf9afbde990cee30fd5b866469&amp;stat=instructions:u">https://perf.rust-lang.org/compare.html?start=62dad457bc73804891c6ac9a31f90de19cbb59a3&amp;end=0d0f6b113047b2cf9afbde990cee30fd5b866469&amp;stat=instructions:u</a></p>



<a name="217498104"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217498104" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> marmeladema <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217498104">(Nov 21 2020 at 11:38)</a>:</h4>
<p>Yeah I looked at those afterwards<br>
It's a mix of improvements and regression, but improvements <em>seem</em> to overweight the regressions.<br>
Even the biggest regressions don't show up as +16%</p>



<a name="217499774"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217499774" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> marmeladema <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217499774">(Nov 21 2020 at 12:26)</a>:</h4>
<p>So preliminary results shows that the regressions are actually in the <code>bitvec</code> crate. My code is using version <code>0.17.4</code> which is quite "old". <br>
With rust 1.47.0, most of the time is spent in:</p>
<ul>
<li>
<p><code>itvec::vec::iter::&lt;impl core::iter::traits::collect::Extend&lt;bool&gt; for bitvec::vec::BitVec&lt;O,T&gt;&gt;::extend</code><br>
whereas with rust 1.48.0, most of the time is spent in:</p>
</li>
<li>
<p><code>bitvec::vec::api::&lt;impl bitvec::vec::BitVec&lt;O,T&gt;&gt;::push</code></p>
</li>
</ul>



<a name="217499835"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217499835" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> marmeladema <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217499835">(Nov 21 2020 at 12:28)</a>:</h4>
<p>Could it be that rust now fails to apply some optimization?<br>
The (really) good news is that, updating the <code>bitvec</code> crate seems to fix the regression :)</p>



<a name="217499854"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217499854" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> marmeladema <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217499854">(Nov 21 2020 at 12:29)</a>:</h4>
<p>That quick analysis was done by running perf record on both versions and perf diff to compare the results</p>



<a name="217500455"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217500455" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> The 8472 <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217500455">(Nov 21 2020 at 12:46)</a>:</h4>
<p>bitvec has its own Extend implementation, I don't see how changes to vec's <code>FromIterator</code> or <code>Extend</code> implementation would impact it unless it does use std's Vec under the hood somewhere. Which is a bit difficult to tell since the type names are the same <span aria-label="thinking" class="emoji emoji-1f914" role="img" title="thinking">:thinking:</span></p>



<a name="217500472"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217500472" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> marmeladema <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217500472">(Nov 21 2020 at 12:47)</a>:</h4>
<p>There is always the possibility that my analysis is wrong :)</p>



<a name="217500520"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217500520" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> The 8472 <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217500520">(Nov 21 2020 at 12:48)</a>:</h4>
<p>did you run perf with -g? then you should be able to get a caller/callee tree and see if std::Vec is involved</p>



<a name="217500525"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217500525" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> marmeladema <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217500525">(Nov 21 2020 at 12:48)</a>:</h4>
<p>I did not, but I can definitely rerun</p>



<a name="217500638"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217500638" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> The 8472 <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217500638">(Nov 21 2020 at 12:51)</a>:</h4>
<p>actually, run perf with <code>-call-graph dwarf</code> and compile with debuginfo=1 (or higher), that'll give more accurate stacks. Otherwise it'll try to use frame pointers which is inaccurate since they can get optimized out.</p>



<a name="217500878"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217500878" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> marmeladema <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217500878">(Nov 21 2020 at 12:58)</a>:</h4>
<p>Yeah debuginfo is already enabled for release builds</p>



<a name="217500880"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217500880" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> marmeladema <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217500880">(Nov 21 2020 at 12:58)</a>:</h4>
<p>I'll rerun the tests later today</p>



<a name="217512591"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217512591" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217512591">(Nov 21 2020 at 18:24)</a>:</h4>
<p>So, in isolation, upgrading the bitvec crate fixes the performance regression entirely?</p>



<a name="217512599"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217512599" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217512599">(Nov 21 2020 at 18:24)</a>:</h4>
<p>Could you bisect versions of the bitvec crate?</p>



<a name="217512617"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217512617" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217512617">(Nov 21 2020 at 18:25)</a>:</h4>
<p>That would narrow down the code change that makes it stop triggering this problem.</p>



<a name="217512630"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217512630" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217512630">(Nov 21 2020 at 18:25)</a>:</h4>
<p>You might even be able to bisect commits from the bitvec repository.</p>



<a name="217513613"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217513613" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> The 8472 <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217513613">(Nov 21 2020 at 18:51)</a>:</h4>
<p>I'd also be interested if there's a case that my change made worse; even if a new bitvec version doesn't happen to hit it anymore.</p>
<p>I have looked at the bitvec source a bit (<a href="https://github.com/myrrlyn/bitvec/blob/develop/src/vec/iter.rs">https://github.com/myrrlyn/bitvec/blob/develop/src/vec/iter.rs</a> specifically) and surprisingly they actually added more code using <code>alloc::Vec</code> since 0.17.4. But they also added a bunch of optimizations. So no obvious culprit.</p>
<p><span class="user-mention" data-user-id="281572">@marmeladema</span>  did you <code>perf diff</code> from stable to stable or from the commit that bisect returned + its parent? The former range could contain multiple regressions and improvements that could muddle the diff results.</p>



<a name="217547209"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217547209" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> marmeladema <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217547209">(Nov 22 2020 at 11:35)</a>:</h4>
<p>I did compare from stable to stable yeah you are totally right that might hides things.<br>
I will try to reproduce the issue with a minimized example in a standalone sharable crate. If I can manage that, that should hopefully help understand the issue better.</p>



<a name="217549585"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217549585" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> marmeladema <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217549585">(Nov 22 2020 at 12:40)</a>:</h4>
<p>Ok, I managed to reproduce it</p>



<a name="217549608"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217549608" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> marmeladema <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217549608">(Nov 22 2020 at 12:41)</a>:</h4>
<p>on a minimized example, the regression is of about 24% for a call to <code>BitVec::extend</code></p>



<a name="217550700"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217550700" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> marmeladema <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217550700">(Nov 22 2020 at 13:11)</a>:</h4>
<p>I verified by comparing from stable to stable. I am now running the bisection again on the minimized example to see if it points to the same commit.</p>



<a name="217551451"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217551451" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> marmeladema <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217551451">(Nov 22 2020 at 13:30)</a>:</h4>
<p>Ok today I cannot run the bisection anymore because some artifacts are missing:</p>
<div class="codehilite"><pre><span></span><code>********************************************************************************
Regression in nightly-2020-09-02
********************************************************************************

fetching https://static.rust-lang.org/dist/2020-09-01/channel-rust-nightly-git-commit-hash.txt
ERROR: Tarball not found at https://static.rust-lang.org/dist/2020-09-01/channel-rust-nightly-git-commit-hash.txt
</code></pre></div>



<a name="217552051"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217552051" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> marmeladema <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217552051">(Nov 22 2020 at 13:46)</a>:</h4>
<p>Surprisingly, even if the bisection has failed, it seems to point to <code>nightly-2020-09-02</code> whereas previous bisection (on the closed source crate) pointed to <code>nightly-2020-09-04</code></p>



<a name="217552177"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217552177" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> marmeladema <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217552177">(Nov 22 2020 at 13:49)</a>:</h4>
<p>Further investigation shows that with rust 1.48.0, bitvec <code>1.17.4</code> suffers from the regression whereas <code>1.18.0</code> (just version just after) does not</p>



<a name="217552229"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217552229" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217552229">(Nov 22 2020 at 13:50)</a>:</h4>
<p>Yeah, there's some updates to that tool that are needed now probably...</p>



<a name="217552239"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217552239" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> marmeladema <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217552239">(Nov 22 2020 at 13:51)</a>:</h4>
<p>So probably something in those commits <a href="https://github.com/myrrlyn/bitvec/compare/v0.17.4...v0.18.0">https://github.com/myrrlyn/bitvec/compare/v0.17.4...v0.18.0</a> has made the crate not hit the regression anymore</p>



<a name="217552309"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217552309" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> lqd <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217552309">(Nov 22 2020 at 13:53)</a>:</h4>
<p>you can probably use the script you used with <code>cargo-bisect-rustc</code> to <code>git bisect</code> bitvec</p>



<a name="217552352"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217552352" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> lqd <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217552352">(Nov 22 2020 at 13:54)</a>:</h4>
<p>now that you've identified the nightly</p>



<a name="217552358"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217552358" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> marmeladema <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217552358">(Nov 22 2020 at 13:54)</a>:</h4>
<p>Yep, will try to do that. But I am confused about two different nightly being pointed at by the bisection</p>



<a name="217552420"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217552420" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> lqd <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217552420">(Nov 22 2020 at 13:56)</a>:</h4>
<p>hopefully the missing artifacts didn't cause the tool to mis-identify the nightly</p>



<a name="217552887"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217552887" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> marmeladema <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217552887">(Nov 22 2020 at 14:08)</a>:</h4>
<p>So I bisected <code>bitvec</code> and it appears that the commit that does not hit the regression seems to be: <a href="https://github.com/myrrlyn/bitvec/commit/f82112243707923a380e5091deab51c29b4f2948">https://github.com/myrrlyn/bitvec/commit/f82112243707923a380e5091deab51c29b4f2948</a><br>
But ... it's a commit that almost rewrites the entire crate so not very useful</p>



<a name="217553433"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217553433" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> marmeladema <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217553433">(Nov 22 2020 at 14:21)</a>:</h4>
<p>Ok I've created <a href="https://github.com/marmeladema/bitvec-perf-regression">https://github.com/marmeladema/bitvec-perf-regression</a> to dump my experiments</p>



<a name="217556072"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217556072" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> The 8472 <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217556072">(Nov 22 2020 at 15:23)</a>:</h4>
<p>That the rustc bisect gives different results is suspicious.</p>



<a name="217558972"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217558972" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> marmeladema <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217558972">(Nov 22 2020 at 16:39)</a>:</h4>
<p>I can try to bisect manually rustc though to be sure. Is there a way to tell cargo to use a custom rustc/libstd? The ones that I would have built from rust sources</p>



<a name="217559074"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217559074" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> marmeladema <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217559074">(Nov 22 2020 at 16:41)</a>:</h4>
<p>Ah yeah just RUSTC, that was easy :D</p>



<a name="217563305"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217563305" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> marmeladema <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217563305">(Nov 22 2020 at 18:30)</a>:</h4>
<p>Ok the <code>bitvec</code> regression seems to have been introduced by commit <code>1d22f75c9f75cad2e408a145861904898ac982dd</code> in rustc</p>



<a name="217563373"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217563373" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> marmeladema <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217563373">(Nov 22 2020 at 18:32)</a>:</h4>
<p>Which is <a href="https://github.com/rust-lang/rust/commit/1d22f75c9f75cad2e408a145861904898ac982dd">https://github.com/rust-lang/rust/commit/1d22f75c9f75cad2e408a145861904898ac982dd</a> / merge of <a href="https://github.com/rust-lang/rust/pull/76030">https://github.com/rust-lang/rust/pull/76030</a></p>



<a name="217566958"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217566958" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> marmeladema <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217566958">(Nov 22 2020 at 19:55)</a>:</h4>
<p>So latest nightly also trigger the regression, I have reverted the offending PR and indeed it fixes the regression.</p>



<a name="217692010"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217692010" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> marmeladema <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217692010">(Nov 23 2020 at 22:32)</a>:</h4>
<p>Apparently it could be a bug in llvm :/</p>



<a name="217730395"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217730395" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> marmeladema <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217730395">(Nov 24 2020 at 09:36)</a>:</h4>
<p>How does one change the label of an issue? The <code>E-needs-mcve</code> label should probably be removed from <a href="https://github.com/rust-lang/rust/issues/79246">https://github.com/rust-lang/rust/issues/79246</a> since I provided a MCVE <span aria-label="+1" class="emoji emoji-1f44d" role="img" title="+1">:+1:</span></p>



<a name="217730930"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217730930" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> bjorn3 <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217730930">(Nov 24 2020 at 09:43)</a>:</h4>
<p>I removed it for you, but next time you can use <code>@rustbot modify labels: -E-needs-mcve</code>.</p>



<a name="217730950"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217730950" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> bjorn3 <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217730950">(Nov 24 2020 at 09:43)</a>:</h4>
<p>With <code>+</code> instead of <code>-</code> you can add a label.</p>



<a name="217730987"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217730987" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> bjorn3 <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217730987">(Nov 24 2020 at 09:43)</a>:</h4>
<p>There are a few labels that you can't change this way, but most can be changed.</p>



<a name="217730995"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217730995" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> marmeladema <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217730995">(Nov 24 2020 at 09:43)</a>:</h4>
<p>Ok thanks! I wish we had some contextual help on issues / PR for that kind of thing</p>



<a name="217731133"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217731133" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> bjorn3 <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217731133">(Nov 24 2020 at 09:45)</a>:</h4>
<p>Even better would be if github would natively support changing specific labels by outside contributors without going through a bot.</p>



<a name="217754268"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217754268" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217754268">(Nov 24 2020 at 13:39)</a>:</h4>
<p><span class="user-mention" data-user-id="281572">@marmeladema</span> I don't think we have an MCVE yet? The repo linked in the description seems to have 3 <a href="http://crates.io">crates.io</a> dependencies, an MCVE ideally should have zero</p>



<a name="217760299"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217760299" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> marmeladema <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217760299">(Nov 24 2020 at 14:26)</a>:</h4>
<p>But how do you measure hardware counters for retired instructions without some help?</p>



<a name="217760360"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217760360" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> marmeladema <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217760360">(Nov 24 2020 at 14:27)</a>:</h4>
<p>and one of the dependency is actually the crate where the regression has been detected</p>



<a name="217760392"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217760392" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> marmeladema <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217760392">(Nov 24 2020 at 14:27)</a>:</h4>
<p>but feel free to re add the tag, I might don't understand what a <em>real</em> mcve is</p>



<a name="217760461"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217760461" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217760461">(Nov 24 2020 at 14:28)</a>:</h4>
<p>Ideally the reproduction of slowdown would be e.g. 10-20 lines of Rust code</p>



<a name="217760572"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217760572" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217760572">(Nov 24 2020 at 14:29)</a>:</h4>
<p>e.g., if I had improved/regressed performance of number formatting, one possibility could be a program like this:</p>
<div class="codehilite" data-code-language="Rust"><pre><span></span><code><span class="k">fn</span> <span class="nf">main</span><span class="p">()</span><span class="w"> </span><span class="p">{</span><span class="w"></span>
<span class="w">    </span><span class="k">for</span><span class="w"> </span><span class="n">i</span><span class="w"> </span><span class="k">in</span><span class="w"> </span><span class="mi">0</span><span class="o">..</span><span class="mi">1000000</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="n">std</span>::<span class="n">hint</span>::<span class="n">black_box</span><span class="p">(</span><span class="n">i</span><span class="p">).</span><span class="n">to_string</span><span class="p">();</span><span class="w"> </span><span class="p">}</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>
</code></pre></div>



<a name="217766281"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217766281" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jake Goulding <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217766281">(Nov 24 2020 at 15:11)</a>:</h4>
<p>One solution is to copy-pasta all the crates into one. Then continue removing things that aren't relevant.</p>



<a name="217766345"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217766345" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jake Goulding <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217766345">(Nov 24 2020 at 15:11)</a>:</h4>
<p><em>Sometimes</em> it's the fact that there are crates that causes the problem, but then that's an important part of the reduction.</p>



<a name="217766388"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217766388" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jake Goulding <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217766388">(Nov 24 2020 at 15:12)</a>:</h4>
<p>I wonder if any of the automated reducers have had any success.</p>



<a name="217767369"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217767369" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> The 8472 <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217767369">(Nov 24 2020 at 15:18)</a>:</h4>
<p>Not sure if it's preferred that way but you could get the perf counters via <code>perf</code> instead of doing it programmatically.</p>



<a name="217774152"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217774152" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> marmeladema <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217774152">(Nov 24 2020 at 16:04)</a>:</h4>
<p>But do we need to reduce it further?</p>



<a name="217775060"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217775060" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217775060">(Nov 24 2020 at 16:10)</a>:</h4>
<p>I think so, or at least, I imagine we'll end up wanting a regression test in either rust or LLVM upstream, and either way that needs to be much more minimal than what we have now</p>



<a name="217775884"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217775884" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> marmeladema <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217775884">(Nov 24 2020 at 16:14)</a>:</h4>
<p>i will try to spend some time on it in the next couple of days</p>



<a name="217793188"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217793188" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> tm <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217793188">(Nov 24 2020 at 18:16)</a>:</h4>
<p>Testing locally, the performance difference seems to be primarily about inlining decision for BitVec push &amp; deref. Marking those as inline removes the most of regression.</p>



<a name="217796141"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217796141" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> marmeladema <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217796141">(Nov 24 2020 at 18:38)</a>:</h4>
<p>But does that mean llvm takes inlining decision based on the name of the types?</p>



<a name="217799204"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217799204" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217799204">(Nov 24 2020 at 19:03)</a>:</h4>
<p>Could be different ordering when making those decisions, perhaps... this is where an mcve would help clarify exactly what's needed.</p>



<a name="217825998"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/217825998" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> marmeladema <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#217825998">(Nov 24 2020 at 23:10)</a>:</h4>
<p>I have reduced it a bit: <a href="https://github.com/marmeladema/bitvec-perf-regression/blob/main/src/main.rs">https://github.com/marmeladema/bitvec-perf-regression/blob/main/src/main.rs</a><br>
but it's <em>very</em> strange, if I remove the usage of serde_json, then I cannot reproduce the regression anymore</p>



<a name="218130471"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/218130471" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> The 8472 <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#218130471">(Nov 27 2020 at 21:29)</a>:</h4>
<p>If it's really just llvm heuristics making different inlining decisions I don't think there's much that can be improved anyway other than providing explicit annotations on affected code.</p>



<a name="218313830"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/Performance%20regressions%20in%201.48.0/near/218313830" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> marmeladema <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/Performance.20regressions.20in.201.2E48.2E0.html#218313830">(Nov 30 2020 at 16:57)</a>:</h4>
<p>Yeah though the performance regression can be really huge</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>